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ABSTRACT 

Program evaluation strategies and techniques based ^n 
materials developed by the California Evaluation Improvement Project 
, were discussed 4% this meeting of the Regional Interstate Planning 
Project^ (RIPP) . RlPP-memb^rs represent the ^tate Departments -of 
Education of. ten western states^ and have mettvperiodically over the 
past nine yeafs to discUss educational issues^f general concern. 
Speeches aft this session included: Education aird 1:h€ Federal 
Legislative Process: News from the Potomac, by Albert L. Alford; 
Purpose and Scope of the Rocky fountain Regional I nterstate ^ Plajining 
Project, by lamar Lefevre; and. History of the Rocky Mountain fi€igional 
Interstate Planning Project^ hb Evalvn Dearmin. (WV) 
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FOREWORD 



Program evaluation stands high todcty among educational priorities. Knowing 
what kind of results the activities in the classroom are achieving has become 
increasingly important. Although the eval u^tion 'of' students ^nd staff is 
crucial, it is in the evaluation of educational programs that the most far- 
reaching benefits lie: program improvements that work for both students and 
♦teachers; conclusions drawn from tiard data that help <iecision makers refine, 
expand, or drop a prbgr^am; and results that have real meaning in terms 9f 
cost and ef f^ti veness^-meaning that can^be communicated to school beard members, 
parents, ^nd others ' ^ """^ 



iDncerned with the schools. 



In an effort to help advance the state of the art of program^ev-aluation, the 
Co^l ifornia Mate Department o-f Education received a developmental q^nt through 
ESEA Title v-C^ Tne Project wa,S' di rected by William H. Bronsojn, he was 
assisted'by Carolyn M. Fowler, and they were backed up by an impressive array 
of p^^acti t loners m the California schools. The results of the developmental 
grant resiUed in th^^oe publications titled the "California Evaluation Improve- 
ment P'^Gject. " 

"^he RwTonal Interstate -laming 'Project staff was apprised of the project 
progress by Verl Snyder of the U. S. Office of Education. In July of 1976 the 
project sta^f invited William R. Bronson to sh^re with the participants attend- 
ing a sen^'inar on "Evaluating foV Effecting Educational Futures' w>iat the 
California Evaluation and Improvement Project was about and the status of their 
act 1 VI ties. , « . ^ • 

He told us that in structuring a group to work with. they had dealt primarily . 
with the s^aj] school districts. Th.eir objective was to raise^kills at 
the-local level in the areas of planning, evaluation, and dissemination. 
For assistance, they went to the county education offices. Their long rang.e 
objective wa$ t^) t^am '\ cadre of trained educational evaluators at all- levels 
to continue ':o '^'i'^.prove ri?fucational corr^etenc i es . Three basic steps were used 
in CQnajctinl? vvor'^shops djring tne developmental stages'--pl an , conduct. 




rational e 

ond'jct 



being you anust plan if you 
v'ou should be able to use 



of 



that 



wi 1 1 determi ne the 



have to conduct, and if 
it. In. addition, what does 
evaluation purpose and require- 
re yo.j doing this'^ If you understand why you are doing something^ 
,re clear what you have to do. In other words, you really can't 
separate prograir olanninq and program evaluation. , ' 

As a result of this overview of the project, it was determined by the (RIPP) 
project staff] to soonsor a workshop based on materials developed by the 
Cjaliforriia Evilyatibn Imorovemen't Project. 

line following Jsominar objectives w^re written to be achieved during the workshop: 
( \ 

*P 1. All pLrticiparts will become aware of ahd better informed about the 
ralifbrnia Ev-iluation ImorftvemQnt Project and its a^pproach to / 



Participants who are program managers from state arvd irr^erniedi ate 
leveM arid those who are already fanfiliar with research and 
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' " evaluation concepts will b*e able to describe, the project to others 

an^ to acquire sl^Jls required to present the materials in subsequent 
workshops. 

3. Participants fror^locdl school districts w-il 1 acquire skills required 
for inipi eref^ti ng or ifDrovinq evaluation designs in their own districts.' 

i 

We. engaged the services of Willidiji Bronson and Carolyn Fowler froni the California 
State Depart'^:ent o^ Ciacdtion, to instrict the workshop participants aba,ut the " 
evaluation Riod^l. ' In additiofi, Reginald Corder and J. Rich.ard Harsh, froni'Educa- 
tional Tesrmg ServtT.es, assisted during the workshop. 

The nater^aiS, covered during the vVork'shop consisted 'of three publications: 
✓ • 

1. Pro^raj:: Eva 1 ua tor/ s Giijde: . A basic manual that leads the participant 

.t'hrou~ah th^' sJteo^^'^Tred, to develop and carry Cut a con^prehensi ve 
^ ;-Uf> 'O^'eva) ja^icn. .'Ptere are chapters on purposes and objectives > 
' r7 and desi gn,;' Wevel opi ng or selecting instruments-, collecting 
.alyzinq data, -and f^l^orting and applying re5ults. 



wG'"^_bofjj' crr>>Pro;jrar. Eva l uation : A recording notebook wj'th text that 
^'er'.^s -^fc a prograir-r'ed instruct iorf in prograrr. evaluation'. 

*3. Evaluaiipn Trainer^' s Guide ' 'A detailed ma-nual for teaching program 
' pvaTi7a"t fori workshops based on the Program Evaluator's Guide, c - 

f 

Other benefits ^derived fro'v^ the workshop were: • * ^ 

1. 'Better evidence with which to discuss programs with school , boards and 
co^irunities. 

2. ''Vjre obiG'.tive and accurate information on program results,* both 
'intTc^pated and unexpected. 

3. Grpater sen-sUivity tg aspects of programs that need refinement. 
More '.ertai'nty that changes will mean improvements. 

Greater assurance that resources will be allocated in the"best possible 
waV. 

At this ti'^.e, Ld-jrationa* Testing Service (ETS) h^ purt:hased the copyright 
privilegps for the ^'nterials from the producers and are conducti'nq a series 
Qf workshoDS to help advance the state of the art of program eval uation^.^ Their 
evaluation improvement pro/]ram, known as (EIP). will also combine npwmaterials 

^ for program evaluation with i nstn.Uj onal programs in their use. A section o-f . 

' this ^Dubl 1 cdti on is devotqd to the future goals and activities pUnned for 
"Evaluation Imprnvempnt Program Materials." ' * 

Lamar L§Fevre, Director 
Rpqional Interstate Planmnq Project 
iievada State Department of Education 
' ' ■ 4055 South Spencer, Suite 234 
Las Vegas', "Nevada 89109 
(702 ) 385-0-191 



INTRODUCTION 

y 

The Regional ^terstate PlanrivfTg Project parti ci'pants have met periodically 
over the last nine. years at conferences sponsored by tepi State departments of 
education to discuss new or topical issues of general cjDncern. The mst recent 
meeting-was held Feb^tfSirY 28-March 2, 1977, in L^s Vegas, Nevada, to discuss 
program evaluation strategies and techniques. 

The R.I. P. P. .Conferences are financed with funds provided^under the Elementary 
and Secondary Education Act of 1965, Public Law 89-10, Title V, Section 505 as 
amended. The/project has been "administered by the Nevada State Department of 
Education under the supervision of Superintendent Jolin Gamble who serves as 
Chairman %of the Policy froard. Other cooperating State departments of education 
are Arizona, Colorado, Idaho, Montana, New Mexico, Oklahoma, Texas, Utah, and 
Wyoming. 

During the process of transcribing, editing, ^nd compiling the R.I. P. P. pro- 
ceedings, ^he editors have sought'to keep the material both brief and informa- 
tive. The resulting report is, hop&fully, a useful record of many thoughtful- 
discussions. . • . 

^ , Dr. Evalyn De^rmin, Co-Editor 

Dr. Jake Huber, Co-Editor 
Mr. LaMar LeFevre, Cp-Editor 
' Research and Educational 

)^ ' Planning Center 

y College of Education 

University of Nevada, Reno , 
Reno, Nevada 



EDUCATICLNA*L TESTING SE^ftVICE 
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'1947 Center Street^ 
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Mr. LaMar LeFev^e, Executive' Assistant 

Nevada Department of Education 

R.I.P.^. Director 
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April 20,. 1977 
S 



Dear Mf. LeFc^re: 

On Lf^r-i] ^\ 'A "ihe L . T. S , 

on iif^^^on , : V. ~.r ' 



:3^^ who atter^ded the L^s Vegas Worksh-^ 
"xp'^ess our aopred ati tor t^h opoor 



tunitv to neet .witr^ y^.j and tne R.I.P P. membershi;;. 

This opportunity was especially important to us because of the 

nign D^iOi'-ty which E.T.S. accords 'to program evaluation. The 

Evaluatiop Improve. lent Program materials developed by the California 
i 

DepartfT'ent of Education have contributed significantly towards 
helping teacher^^^rinci pal s , and program managers within the 
state develop .improved evaluation "skills* We are pleased to have 
been selected by the California Department to'extend their work 
by publishing and distributing the ,E.I.P. materials on a national 
basis and appreci ate -the interest and support of the R.I. P. P. 
Fnembership in providipg- advance information about their 
availability. \ . . 
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.T.S. in/ a 



, The E.l.P. materials dte now available from E.T.S. inr a somewhat 
revised formart. Later in 1977 w'e expect to add additional com- 
ponents including instructor's transparencies, a slideytdpe 
orientation presentation, a series of tapes ii^udioca^sette form 
to serve ^s aids for the program eyaluator, and-x^an additional unit 
of instruction on the current Title I evaluation models. 



The hospi^tality and courtesies yob extended to us in your role as 
hostiwere superb. • We than^ you and look forward to another meeting. 

Very truly yours , 

Reginald Corder 
Professional Associate 

Research and Advisory Services . ' 

for ' ^ ' I ^ 

Patricia Elias., Director, Research and Advisory^ Services, Berkeley 
J. Richard Harsh, Director, Los Angeles Office 
Theodore Storlie, Professional Associate, Evanston 
Wesley W. Walton, Executive Associate, Princeton 
Patricia Wheeler, Professional Associate, Berke/ey 
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PROGRAM ACTIVITIES 



Mori? 



. February 



8 00 k.ni. 
9:00 a.m. 



10:00 a.m. 



n-40 a.m. 

1:30 p.m. 

3:00 - 3:20 p.!.' 
3..?0 - 5:00 p.m 
5:30 - 6'30 p.m. 

Juesd ay , Ma rch 1st 
• 8 30 a.m. 

9:00 a.m. • 
^ 10:30 - 10:45 a.iiT. 
10:45 - 12.00 noon 
12:00 - 1:30 p.m. 



1 :45 p.m. 
. 3:30 ' 4:00 p.m. 
5:00 p.m. 



Re'gi strdtion (Shiloh Room) 

Greeting 

J oh: Gv\ble 

Ucva'li SupeHyitCKdc?it of Public 
iriS ti[*iiJtion 
Introductions^ 
La^lii' Lrteore 

Divide into two working groups 
Team 1 -(Biloxi Room) 

Carolyn Fovlcy ::nd J- Richard Hareh 
Team 2 (Shiloh Room) 

William drcMson ayid Reginald Carder 

L^nch on your own ^ 



\ 



Reconvene group sessions 
Team 1 (BiTt)xi Room) 
Team 2 (<ShT!oh Room') 



Break (Biloxi Room) 
Work' Session 

Personality Evaluation (Shiloh Room) 
(Comp. Showboa^) , ^ 



Coffee 

^Ses&iofi begifis 
Break 

Reconvene work sessions 

Scheduled lunch 

Speaker: Dr, A I ford 

Associate Comnissioner for 

Legislcrtion 
U, 5. Office of ^Education 

Reconvene work sessions ^ *^ 



Break 

On 'your own 



Wednesday ^_Jlarc h_?nd. 
8:30*- 11:15 a.m. 
11:15 - 12:.50 noon 



Work Session 
Wrap-up Session* 
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from the ^al^uation Manual ^ 
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Richard 'Harsh S Carolyn Fowled 
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Workshop participants from"' Arizona 
"foby Jalowsky on -tKe le.ft^ ^H^rma 
Hightower and J\oy ^ '^lll , 



I 





"'V 



■ / 



William Bronson':of the^ 
Department of EduoaWon, 
lectd^r, makes a point on 
chalkboard ^ . , 




Denis Graham of the "^^vU'la Depm-'tmcKr 
. of Education and Don RijhavdSy IJtah 
\ ' • delegationy along with fellow parti- 
civantSy follow tha Workshop 
proceedings \ 




Shirlee WedoWy Nevaday^ Joins Reid 
Bishop (on her right) and Idaho 

delegation 




Dr. (fhdrles' San-lnayiriy far right y 
with the' other Oklahoma -partici- 
pants - ' J . 





LaI42r LfFevrey R.I.P.P. Troj-ect 
* Directory \Nevaday and Reid Bishopy 
IdahOy discuss a point wi-th Ifovkshop 
participants 
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EDUCATION AND THE- FED^AL LEGI^^TIVE PROCESS: 
NEWS FROM THE POTOMAC 

Dr. .Albert L. Alford 
' Ass'ist^nt Commissioner for Legislation 
^U. S. Office of Educatio/i 



Authorizations and Appropriations - 

Fpr t:he past A2 years I have been .in charge of the Office of Legislation in the 
U. S. Office of Education. As an assistant to the Commissioner I coordiiiate the- 
activities, involved in developing legislation, in •developingJ|estimony--in a ^ense 
selling itf on tfie hill--not in t+ie lobbying sense b^t in the*reparation of^the . 
technical materials needed for testimony and bagkup 'for the administrative pro- 
" posals in .education^ 

• In the Congress, the elel9ientary and secondary programs are coming up for renewal,, 
,We *re now looking at those particular programs more intensely. I work only with 
the authorizing 1 egisl ation which is qui te different from the appropriations. 
Congress first has to grant the authority to set up and carry .out programs* 
Getting the authorization, however, doesn't mean anything until you get the 
money to run it. You- have -to go through another set of committees, and another 

j|brocess to jget appropriations that really make the^programs go--the dollars. 

ritual ly most people probably have a greater interest in *th§ appropriation process 
than thSy do in the authorizing process; but you can*t have an appropriation it 
you don't have an authorization, so they are both equally important. 

From .tTie ap|5ropriations end, and that's all we've really done much with at this 
time, ^ the news is pretty good. The Carter, administration sent forward their 
budget early last wefek. It increased by about one and Dne-half billion doljars . 
over* the Ford budget the amount for education. And not only that, and more 
imp6rtSntly perhaps because the Ford* budget actual l>\rQpresented an amount below 
what we were already scheduled tolget in*1977, the parter'budget represents about 
a $600 million increase over the 1977 ^udget, which\begins next Oc^ob'er 1 and 
ends Septeonber* 30, 1978. I have no indication that 'the Cbng|^e5s .wi 1 1 give us less 
money than vye ask for. In past years. Congress has'tended tb'give us more than 
th€ President^asks for. Certainly we would not anticipate a percentage increase 
as we have had for the last eight years- because we are starting out withM much 
-higher base, but there may be adjustments tha^ the Congress will make on T:he 
dollars even beyond wha^the President has'asked for. 

' In the Irlpact Aid Program the Cartel* administration took the same view as .the 
F^d administration which had the same view, is the Nixon administration and the . 
,same view a^;the Johnson administration: that the program is too big and needs 
to be cut back. Although that view has been -consistent since the early^BO's, 
Congress hasn't paid any attention to it in all these^years. The* program still 
' keeps growing. We will again go through the exercise'of determining whether we 
can cut the Impact Aid Program. 

\r\ general we^ are off on'a positive note in the appropriations eftd. I hope that 
we will not have the kind of hassle that we have had for the la^t eight years 
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in fighting the general public concern to cut down or to hold educational expen- _ 
ditures below budget. I don't know that we. can anticipate ir> the executive 
budget any large-seal e 'i ncreases of the magnitude that we are talkiag about, 
this year as we come up in future years because Pres.idefit Carter is ^very con- 
cerned about the economic situation i^^ this cotintry. He's very concerned about 
inflation and unemployment. Hjs major emphasis will be trying to solve sdme 
of ♦these economic problems. If tFiat can be done, domestic programs will be 
increased or adjusted as they need be. As a matter of fact, we really can t 
anticipate any major, large programs in the, area of education unless t^f^^rogram 
is designed either to reduce nfl ation or. unemployment or both. The emphasis 
will be on the kinds of programs that affect the economy. 

Atypical Fupdi ng Sourc es . ^ ' • 

There are a couple of programs educators need to keep alert to. As a matter 

of fact there cfrg some areas, to mine for funds that are not in what are typically 

educ/ational programs. The PuBlic Works Act passed last year provided some S2 

billion in potential appropriations for construction. The schools have fared 

well -under tbis one, coming ud with about 20 percent of the money national ly. 

Programs like these have to be worked through areas other than the normal educational 

hierarchy. The public Works Ait is operated through the Department of Commerce. 

Another Public Employment Bill was tacked on as Title VI to CET,\j the Compre- ^ 
hensive Employment Tra^ining Act;, which was designed to create public jobs in 
areas of high uaemployment. A few States have used this^funding to develop- 
kindergartert programs they needed which then made markets available for the 
hiring of unemployed teachers. So there is some potential even in this area. 

> * 

Budget Refo rms Act and Ex pi ration of Authori zation 

Much educational legislation is now e)ipiring. Let me just run over a list of 
the pieces of legislation that are up for renewal at this time. In the Uttice 
of Education we now operate under the new Budget Reform Act which was passed in 
1974 We have to get our recommendations up to the Congress on a fixed timetable. 
The Congress, the authorizing committees, also have to take their actions by a 
fixed time in order to get the bills out on the floor to be considered. 'This 
requirement poses a r>eal problem for us in terms of >ead time. The law, as 
now set up, requires the authorizing committees of the Congress to report out 
by May 15th all legislation which will expire on September 30th .of this year. 
We in turn have to have our recommendations through the Congress and 'through 
these committees a year in advance of May 15th; so we had to have ours in by _ 
last May 15th on the things that are expiring this September 30th. On May Ibth 
of this year we will submit recommendations .for the legislation expiring September 
30, 1978. The pr;oblem Vtes in backing up from those- dates to allow adequate time 
to'evaluate the program and determine what shoul.d»be changed. 

We're getting' lead times of two to, two and one-half years in our internal process. 
That means evaluations have te> be sjtarted three 'years or so in advance of the 
time that we need the information , to deal with. So we may be talking in terms 
of five-year lead times to really effectively use the results of evaluations. 
Thos.e deadlines and those dates 'cQwpel us to net our,act_ together. 



We're having some problems now because of the change j*n administration* • We have 
the Secretary of HEW confirmed arid the Undersecretary of HEW confirmed, and** 

- ♦ that's it as f'ar as HEW is concerned. The Controller has been asked to-^tay 
On, 'therefore,, the Controller's Office is in place^and operating; but all of . 
the other assistant secretaries within the department and the Commissioner of 
Education and ^ Executive .Deputy Commissioner and-afl of the five deputy 
commissioners a% still scheduled to be appointe^l.^ In^'a number of cases' the 
•deputy d)mmissioners have yet to be named. In none -of these cases have the 
papers even gone forward to the Congress f4Dr the confirmation hearings. The 
nominees are in Washington */ofking qji a consultant basis, they are involved in 

• . developing 01)1 icy, but they can't reaMy give ar^y di rect- orders for doing things; 
They are wooing on a very informaT^^asis, -which makes it diffigult to undertake 
the tough policy decisions in many areas. ^ • * , ^ 

For example, we need to review the^whole area of ^el ementary and secondary educa- 
tion because those programs arB due' to expire on'September 30, 197^^.' The .Carter' 
administration has, at.least tnseneral .terms, 'rejected as an ov^r^l] proposi- 
tion the consolidation propos^TT^of the/Nixon and^Ford administrations. They 
have not rejected looking at (some possiy ^ cojosol idations at some point, along 
the way, but t^ey rejected th^^^overal 1 , wholesale Concept that was uSed by the 
past administration. We have m look at the individual programs and make some 
determinations about them in time to -send the recommendations to the Congress 
by May 15th. Because so many of our people ^re not yet on board, it will be 
difficult to render these decisions. * 

We have some more rritical "situations. Ttie Library Services and Construction 
Act^ for- example, which deals-with public libraries actually, expired las't /September. 
We have' an automatic one year carry-over authority for every program adffli/ii'stered 
by the Cormii ssioner of Educati-on, so we st-ill have authorization for it;/but we_ 
do not go by that one year carry-over irf'terms of our deadline 'data for /submitting 
the recommendations to Congress. _ If the -program is not to be absoluteli^ expired 
By this next September, "(^bviously the Congress is going to have to mov0;. Tpe 
House didjsass a bill last-year extending Jhe program, but'the Senate 'oeyer\got 
around to it. The House has just rep(3r*ted out a bills and they held hearings, 
and we were'invited to come to the hearings;'" but since we did not have a posiV . 
tion, we had to ask not to go forward.-, ^e dicjn't want to take-a negative [K)siV 
tion, but we didn't have any other pas'i-tion at that til\ie. The House has moved \ 
^ ahead and will undoubtedly get t^ieir bilTpassed on the floor, and the Senate 
has scheduled their hearings for March 9tti. ^If we can now get our pol i cy 'togethl^r 
in time for next week,' we will go betore the Senate and testify on the Library \^ 
Services Construction Act. 

In addition, of cou>se, a number of bills have rfot been considered pa'rtiajly 
because as far as the administration was ^ncerned we had to have them up last 
year, ancJ so we did; but most of ours were involved in'the consolidation. We 
simply threw them alTin the consolidation-package and didn't s&y anything ab 
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;imp1y 

th*e individual programs 



jtipn-pacKage ana aian t s/ay anytmny about 
tjfk Environm^al Education Act, the .'Alcohol and Drug 
Abuse Education Act,'a-nd /Tl.of the>discret1onarji'programs in |he Educational 
Handicapped Act. The Part D,' the State^Support Prograoi under the Educational 
Handicapped Children Act, is now permanent authorization. It will nx)t expire. 
Therefore, we-^o not have to .deal with.it, but all the individual parts dealing 
•with the research and training .and speCial kinds ot projects do have a limited 
term of authorization which will expire at the end of this year." Of' course, 
the one year automatic extension would carry .over, so there's no urgency ^if 
.this one does tlrag past October 1. 
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Title VI, jfefie National Defense Education A'Ift, which is area studies .and language 
development, thp ^ucational Television Broadcasting' Facil ities and the Head 
Startirbgram, or the Follow-Through Progra\n that also includes the whole Head^ 
Sta>>t area of operation -expire in 1977. We-need to make some individual decisions 
o/these in terms of r'eview from the point of view of the Carter" administration. 
'The legal aspects of this are taken care of 'because the administration did send 
it for.wafd last May, but a review process, is needed. 

' * * * 

The 1978 expirations, upon which we are- going to. have to make our recommendations 
the"fir«t tii;ie this May, include the whole Elementary and Secondary_ Ed*ucation 
Act, aLl nine fUles of that ^ct--the big Title I, and the consolidation titl« 
which consolidates Titles Ii; III, and V and most of VIII, and Title IX, the 
Ethnic Heritage Prcqram: The Adult Education Act also expires. We have made 
dur recommendation to eliminate Part B of' the Impact Aid Program, and that has 
gone forward. All the provisions of the Indian Education Act expire and are up 
for renewal. This area becomes important because ^ considerable amount of money 
flows from the Indian Education Act into J:he western part of the United States. 

0,,r Sof-ie^ pVoiects AuthoVity includes a discretionary fund for the Commi 5s'ioner 
0^^ Edu.-dMo- Th'^ law reauires us to put at least half the money into the desig- ^ 
n^^ed pro-ia i ^na'. ^on'jress has packed out, and these include the Metric System, - 

Giftp^' ar-^ T;^v-.n^pH "hildren, tommuni ty^ Sthool s Career Education, Women s 
Eguit,/ i7/"rrjc.?M V,. and the Arts in Education Program. All of those need to , 
^be re'viewed to dete-'-.n-ie where we are going with them. 

As you can sep- we have a "very substantial number of items up for expiration. < _ 
My office has 'a Tot of work cut out for it this year. These authorizations 
probably 'wil^^p considered over the -next two years or year and a half because 

Congress has h^d a tendency, since about 1970, to ^^hATr^ Hanal °Amexid- 
•legisiatior., an c^ibus bill, every two years We h^^^the Educat onal Amerd 
ment-s of 1970, 197?, 1974 , and 1976. Most of- these expirations will await packaging 
inta the anendjrrents of 1973. - • ^ 



Evaluations 



X" Although I don't aet involved in actually making evaluations, theoretically we ^ 
^ are supposed to use them, a^d the Congress has placed S great emphasis on eval- 
. uation over the years. As a matter of fact, when. Congress passed the large 

- FitleA program in 1965, annual evaluatioBS were required. We had not requested 

- an anSal evaluation tjecause we "had th^ht' more in .terms of a ^^^^"f ^^^^^y^J^r, . 
tion in order to give th6 program an o^Prtumty to be developed. We really. dTdn 
tMnk we cou^d get much in te'rms of ev^ation the first few years as the program 
was too new".- ' But Congress said they wanted evaluations immediately. 

One advantage to that is getting people star.ted on evaluatioh ^ou have got 
to plan your evaluations and establish a baseline. You. have o build hat in o 
the program. In a provid,ential type of evaluation, you hav€ to go back and p ck 
up the data and "try to recreate it. This poses real prob ems when ^""J^body cal s 
you to task on a true statistical analysis. You^are usual ly unable to P ^ 
that your results ffiean much of anything. You need time to Set up an evaluation,^ 

to plan it, and to get it inti^ operation. 

f 
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My ofltice works ve^y closely with the office in charge of planning and evalua- 
^ tion :in the Officeiof Education. We are really part of the same office conglom- 
■ eratei We try to ^et up a timetable to allow them to let their contracts and 
pla>kfor the evaluations we will need so that th$se can be used 5s part of our 
planning process or the legislative end. We have really .only started in the last 
three or four years to ge? this kind of intensive planning, and J^J^asn't come 
" to fruition yet. i • ^ " 

] ) ' 

The National Institute of Edu-cation has a SlB^nillion contract over three years 
to conduct studies jin compensatory education. They published a preliminary report 
' - at the' end of Decem[)er. The other reports will be coming out toward the end of 

ttiis year, but they, will not really be av^a\lable for us in terms Cf our transmit- 
' tals to Congress oniMay 15lh a-S far as tne- recommendations are^ concerned. 

' Evaluations, of coufse, are very .dif fi cul t to apply. Even if we start getting . - 
• • all of them flowing ;into ^us, I don't know whether we will be^able to use them 
or not. I've only l^ad one instance of trying to use an evaluation in the 12 
years I' ye worked wfth programs and that was on a proposal we made-in the area 
• * of 3cho6l aid. We^cialled for' the eliminiation of the large-state grant program, 
i' ;:onyerting it iRto a» di sj^^etionary program which could concentrate on some special 
areas where we^ h^d fl)urrd that it was effective. From the very beginning, we had 
a strong eval-uation Component, ^nd as evaluations beg^n to flow out of it, we 
found, fo>" e/ar-.ple, the hujnan re-lations programs in the school districts were 
quite effectives Bet^axjse they secured positive results which seemed to hold 
uD^under all' of'^-the natistical analysis,, we thought we should really go for that 
one area'and spend lass mOney in the di scretionary^ porti on. 

WheTi you' try to. propose 'less expenditures of money for programs and use your 
. Revaluation results a^^your backup fpr this, you frequently find that the e\ialua- 
' tjion results are *i gnot^ed .because , in many cases, the Congress does not want less 
-money to flow, the lodal scho61 districts do not want less money to flow, the 
States (to not want le^s money to flbw. A very, very convincing case has to be 
^ made in the use of eva;luation results to bring about the end of a program. 

Of toursg, the Vong haul advantage in the u'se of evaluation is to improve the 
programs.' Bi^t, if youUre not really tPfinking in terms of cutting off programs 
or abolishing prograriis you eliminate that concept, tou assume that even if 
the nature 0^ tKe« program rs ahdjusted slightly, the money is. still going to* 
flow. There is' a need Ifor alVth^ money that we are now spending in education, 
'but let's adj:jst it ah4 get it into channels where it is going to do the most 
^ Qoi)d. Vhfere, I think wfe may be able to use th^ evaluations to some effect. 

J ' For example, the 'l974 af|]en'dments called specific411y for a look at a change in 
formula for the Title I |program--to mo\ie it from a poverty bas^ distn bution to 
\. an educational deprivatibn base.' Put in those terms, you measure the educational 
abilities of the individt/al students, and you flow money in to remedy the 
^deficiencies in the mdifyidual students. When we designed that program we 
corFelatecKas nearly as /we could the venous census data that were available. 
It was fairly crude beciuse we correlated educational achievement with poverty. 
^ It came out very high, jl "t'hink it would hold true under any kind of assessment / 
you ma'ke. There is a very high correlation between poverty and educational w 
. deprivation because thei two thipqs go togather in the system. We did have the 
poverty measure; so it-f/as very easy to sqnd money out on that basis. Then we 
told the schools, after! theV got the money on the poverty measure, now look'on 
an indi^vidua; basis at Jyour students and see what needs to be done in educational 
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terms. You may need a comprehensive reading program, you may need somethi^ng in 
math, you may need this, or you may need that, but we' 1 1; ] eaveM* t to you to 
determine what you need in those areas. • .We asswe that when we get this money 
down to spools' ir an area that is poor, very h^gh numbers of students* are going 
^to' need tms extra hel;^ :n^terr,s of educationalJ'Support. 



education deprivation and ^try 
i^ds of problems. You have a 



If, hawever, yo^i tr^y tj.jse a r.easui^e of actual 
to put that int^o a ^"or^^uia, you get into all M: _ , 

Droblem of test.inq, o^r esta'' i i shing- tests that ere recognized and accepted-by 
everybody ".^hen you^ try .to do that on a national level., jt betome? very impor- 
tant because th^e new increase in the Carter proposal ^of $2,635,000,000^ proposed 
to start in 1379, and tha't is a..subktantial^p1ede of change r^o matter what way 
you look at it. It becones very critical wha*t nieasure is used in distributing 
the money.' -opetuViy, we will be able to use sorAe evalua^tions of programs now 
being conducted by NIE to-see whetter itMS possiUe'to use another measure, ^ven 
though" tlj^y were' only looking 'at the d^'.stributions\ in, a school 'district, between 
^'chools.* Sore ;^^yoj ^ind you. can, ther^e wilT^be pressure on" the part 'of -some 
Con';ressmen to -oye o^t into other areas; b^jt "I't vdoes convert the program., 
•^'hat it rPd'l^ -^eans"is ttiat you-.give money for any ^itiild,^ whatever >the income 
level M^.of^'ts; to help tl^'em; to remedy 'their deficiencies. ^ ^ 

All. of yo. oe. /<-'r^;n- o>> th^s in yoJir individual States, and, th^s will' be 

-"low-sn- ,in:: t^e s/ste:^> because we have the Congress' interest in the State and 
"lo^-al evaluations.' "ney nave ^ade thafquit^ clear to u^.. We're'to utilize the 
bulk 0^ oi/r -onie3 i^? the evaluation field for technical ass-lsta'nce for the 
creation State, and local evaluation programs.' What ^e have to do .under: those^ 
kinds c^ syst'^-^s,* is -to^ get tnem reasonably ^comparay e so- that the infprmation ^ 
that flows can be Xilized nationally. This prgblem -always exists wheri you^ 
move ojt t^j' the -seoarate units'. Many of- oar evaluatiftn'peqpl e/tend^ to favor ' ^ 
^he na^^^oral '■y-"'^aeted Evaluations because they control the data bas>e, it's : 
p^l colle"^^:^ in one wav, and they th^ink they can get 'stattsti cal ly better results 
That^may re tr-.p*,' ^"^nd it undoubtedly is true, but whe/Fi^r true^or not the Congress 
said "Wp'^^ -oinn:to do it the other^way," aftd' so we are g®ing to have to 
'ne ^.rnrpss and t^/ to do someniing about being able to feed this whole 
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iini ifito a networ^K .o^> evaluation. ^ ^ 
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PURPOSE AND SCOPE OF THE ROCKY MOUNTAIN 
REGIONAL. INTERTSTATE PLANNING PROJECT 



* by ' I 

Idmdf Lefevre • \ ^ - ' 

Assistant Super-intendent of ' 

Public Instruction - • , » 

Nevada Department of. Education 

and ^ 
^ Director 

\ ' Rocky Mountain Regional 

Interstate PI ann jng ^Project - s ' ' 

'On July 1, 1974 the Nevada State Department of Education assumed the adminis- 
tratipn of the Regional Interstate Plannir^g Project. Since that date, the 
Project Policy Boar-d has sponsored nine major seminars ptealing with ediiaational 
topics that were, major concerns of the, participating States', The meetings^ have 
been' attended by State 'Department -of'Etlucation personnel, staff from local school 
districts^ i/mveV^i ty- prof essors , staff Jrom the U.S, Office' of Education 5nd 
regional offices.^ In addition we have -had ^participants represnting local 
and State Boards of Education and other Interested jcitizehs. 

During the tirne the Regional Project has been operational, many change^have 
occurred. The one change that is most noticeable, and the one th^t is least 
talked abouti^ is the attitude of the State's representatives' when they share 
program accomp]>emiente, methods, and ideas that brijig measurable if'esufts. 
Participating State staffs have *devel oped •a communication rapport. Ihey 
listen and sha^^e i.deas, utilize and refine information- acquired-about educa- 
'tional models in^their respective States, and integrate such models 'into their 
program activities: 'As individual State agencies, they no longer re-invent' 
the wheel each time they respond^'to an educational marid'ate. 

Beyond the fornial nieeting times, the- sta^f members from the different State 
agencies hove made professional contacts thr'bugh the seve^r*al regions; and in 
>the U.S. Office of 'Ed jcat i on . Contacts made by individuals with their counter- ^ 
mparts in the va rious^ States, and federal offices have become a valuable as^et. 
ThrpugK these contacts, the process of responding to many educational concerns 
has been accel erated--especi al ly whare federal law and regulations ate involved. 

The national netowork of Interstate Projects^was conceived to achieve the 
following objectives* ^ ^ ' . / 

1. Identify, analyze, and work cooperatively on current and emerging, 
educational problems of'cQmmon comcern to all or' most o^ the 
participating States a^nd ,terri tori es. • 

' ' J 

2. innovate, pilot and develop methods w+iich can be used by State' ' ^ 
'Education Aqencies to strengthen their leadership and service 

capab*! Vi ties. 

3. Plan, develop and implement staff development programs -which will 

* improve State Education Agency ^rsonnel skills in solving identified 
problems. 



"4. Improve and stimulate intercoRimunication and dissemination of prom- 
ising sdlutions of educational problems among par.t ici pati ng agencies 
. '^and' the -U. S. Office of Education. . ^' 

To cpmplemrent these national goals* tjie R.I. P. P. Pqlicy Board developed the 
following goals to^ be achieved*, while conducting various activities: 

1. To^ increase the know-ledge of .selected SEA personnel relative to 
predetermined educational .topics to the extent that knowledge 
gained 'may^. be useful to th^ participating, SEAs. 

2. , To share current Activities within the SEAs relative to the ^ / 

selected ar^eas'^of common concerns to the extent that the parti- ' , ] 
cipating SEAs may find the ififormation to be useful. 

* 3. To provide materials for-di^semination to-be used by SEA 

personnel and others not in- attendance and for future refererlce. 

» 

. 4. To alert selected ^ersonn^l about emerging trends in education 

-and to provide assistance upon" request s'o that participating SEAs* 
'will be in the forefront of new developments. 

5. .To maintain communication vfith the USOE Division of State Agency. 
Cooperation. 

As one reads the individual publications produced by' the Project, a result of 
a semijUar activity, i*t is ^evident that the national network gdals-and Regional 
Prpject goals were attained. - ' ' - 

How that authorization for funding the Project has been discontinued, and 
Project ^activi.ties' will, no longer be scheduled- as .in. the past, a few commenda- 
tions are in order. ^ - 

The R.I. P. "P. Policy Bocird and Project staf-f woutd like to thcink'the U.S. Congress 
and U.S., Office of Education for making this type"'of activity available for . 
.the prof essi.onal growth of the ,many'individual s who have been involved in the 
Project during the^yec^rs it^has been^operational . *The Project staff would ' * 
especially, like to tliank Verl Snyder and Dr. Al White for their many-faceted 
quality assistance, operating out of the'Division of State /tss^stance, U.S. 
Office, of . Education • , 
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HISTORY OF THE -rocky' MOUNTAIN 
. REGIONAL* INTERSTATE f LANDING PROJECT 
(RIPP) 

Dr. Evalyn Dearmin 
Senior ' Research Consultant 
Research and Educational Planning Center 



Estarbl ishmen t 

When *tbe Rocky Mountain Regional Interstate Planning project was origi- 
nally^ formed in 1968 as the Interstate Project on Planning and Program 
Consolidation, it consisted qf a consortium of six's'tates: Mont.ana, 
Idaho, Wyomiiig, Colorado, Utah, and New Mexico. The States of Nevada 
and Arizona joined in 1969, after leaving the. Pacific RIPP. Awo other 
States, Texas and Okla^ma, also joined in 1972, bringing the^ present 
membership of the Rocky Mountain RIPP to a total of ten Stages. 

Th§ four States which- wi thdrew from their regional -projects, to join the 
Rocky Mountain consortium did so for several compell ing reasons. They 
felt ^ more common bond of concern with the other meqiber States of the 
Rocky Mountain RIPP. The problems of these States with relatively low 
.populaticMi densities and larger geographical expanses were^more compar- 
able to .their own. Furthermore a greater cost ^savings could be realized 
if ten ytates combined into on^ group rather' than several States meeting 
in two or tfrree smaller groups. 

The current membership of the Rocky Mountain RIPP also includes four of 
the USOE regional offices: Region X in Seattle, Region IX in SSin Fran- 
cisco, Region -VIII in Denver and Region VII in Dallas. This comprehen- 
sive representation serve's to reinforce communication among the Western 
Staters. 



Growth and D^evelopment 

' ' ^ • ' . • ' ^ 

Initially^ tjje adrriinistration of th^ Rocky Mountain RIPP was, situated 4n' 
Denver under the aegis -of the Colorado Department of Education and the 
directors'h'ip ot Arthur R. Olson. A Board of Directors comprised of the 
chief State sctooI officer of ^each partici^pati ng State, or his represen- 
tative, has s^erv^d as the coordinating or planning agency from^j|j^out- 
set of the project. Early each year topics of commori concern a^PIg the 
ten participating States are identified and plans ar'e made to develop 
seminars or wprkshops appropriate to the topics of greatest interest. 

From 1968 to 1970 few documents were published by the Rocky Mountain RIPP 
The discussions were of. such a kind and quality, however, that they 
deserved to reach a wider" audience; and jn 1970 the first/continuing 
.series of seminar proceedings appeared entitled simply "A Workshop Report 
The subsequent publications a-r^e ample testifribny to the increasing val<e " 
of the workshops and the growing diversity of issues confronting State 
departments pf education. 



Early Emphasis On Accountability 

Recognizing a growing need to render themselves more accountable to the 
public and to their clientele, educators participating in the two RIPP 
'.workshops held in ^Denver, Colorado and Baise, Idaho in 1970 addressed 
themselves to the process of managing more efficiently by establishing 
objectives. Management by objectives iMBO) was the pervading theme of 
both conferences, and attention wa.s focused upon the design and imple-^ 
mentation of planning; programming, and budgeting systems (PPBS) for lo- 
cal and State education agencies. . 

The accountability issue was pursued at the next three conferences held 
in Phoenix, Arizonii Reno , 'Nevada; and Helena, Montana in March, April 
and July> 1971. federal mandate to "give each student the compe- 

tence he needs" dojninated discussions of means to attain that end through 
changes in traditional organizational structures and through performance 
contracting^^ ' ^ , 

The Phc^nix conferenrce attempted to acquaint participants with the spe- 
cifics of ^performance contracting and to' provide them with some guidance 
in implementing performance contracting in their districts or regions., 
The conference at Reno investigated two questions related to accountabil- 
ity!^ schools guarantee the acquisitfon of basic skills?'^ ^d 
"Do the schools use the most economical means to that' end?" Several 
State department of education representatives discussed various systems 
for attaintng accountability throug^i performance contracting, manage- 
ment control,^ and appropriate evaluation procedures. The hazards and 
limits of thesp proposed rfiethods were^expl or;ed as well. 

The subject of Management by Objectives was "revisited" once again^at 
the Montana conference. The participants had moved from definition of 
abstract managerrient concepts- to a need for concrete information appli- 
cable to their specific circumstances. Nevada ^representatives offered 
a case study of their past experiences in SEA management, Oklahoma rep- 
, reseritatives prof fered' advice on developing a records and information 
management system, and Texas representatives submitted a State level 
management assessment system. The presentations were a ^ref lection of 
the professional growth of the Rocky Mountain RIPP which had elicited 
issues of commfin concern and examined them fully from concept to practi- 
cal appl ication. . •' ' ' - '/ 



. • SDE's^Seek Stronger Role in Public Policy-Making ^ ] 

In- 1972 the attention' of th^ consortium was directed toward providing 
future^leadership-. Rather, than being reactive to the public demand for 
greater accountability, the SDE's were searching for, a stronger role, in - ^ 
making publ ic 'pol icy. Problems attendant* upon forecasting and improving 
' decision-making came insistently to the fore.^ The historical handicaps 
. ef sea's in developing- pol icies and the politics i/ivolved,in planning 
were stressed at the semirfar held in Las Vegas, Nevada on February 29- 
March 1. An awareness was 'evident that substantial numbers of individ- 
uals' and groups were interested in participating in the public decision- 
making^process, particularly as it .affected education. The means SEA's 
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could use to Involve citizens, and other governmental entities in system- 
atic ed.ucational planning werejexamined assart of the new leadership 
role SEA^s had to assume during a time of vociferous public controversy 
^about education. 

Controversial Issues'Become Rl)^P Staple Fare 

During 1972 William G. Grimsle^, of the Colorado Department of Education, 
became the Director of the Rocky Mountain RIPP. In Denver, Colorado, at 
their^ July 18-20 meetirJo,' conference participants grappled with> an ex- 
ceedingly contro\/er5ial^ssue--the pej^formance-based teacher education/ 
certification movement. A statement made by the seminar -correspondent 
Guilbert C. -Hentschke of Columbia University Teachers College reflects 
the^circuRispect approach the RIPP conference planners were taking in'.re- 
gard to these kinds of issues: 

As a group y the indwidual- presentations" tended to pull 
4 against each other. The strong points in theory are bal- • * 
anced by the , problems encountered in implementation. Tes- 
timony about possibilities is seasoned with caution and 
even some skepticism , Yet, is this not about the only ' 
way to address a subject 'which is clearly emerging but • 
whose- specific future is zo date unclear? 

that the caliber of the presenters was very high, including Karl Massanari 
of the AACTE, Wendell Allen of the Washington Stfite University Department 
of Education, T>headore Andrews and Helen ^tartle of the New York State' 
Department of EobK^rtliDfi, D.D. Darland of NEA and Sandra Feldman of UFT', 
indicates the care being .tak^n to* assure that the selected issue would 
be examined fully from a variety of professional perspecti ves--the teach- 
er training institutions, the teacher unions:, the SEA's, and educational 
researchers. ^ ; 

In 1973 the Rocky Mountain RIPP turned its attention to the most^'^^ely 
publ icized current Mssue in education," school f inance^reform to achieve 
equal educational opportunity. The terminology of the seminar presenters 
indicates the dimensions of legislative and legal intrusion into school 
planning processes. The Serrano v. Priest decision in California, pre- 
venting the quality of education from being a function'of district wealth, 
and the Rodriguez y. San Antonio Independefit School Qistrict case, which 
found wide. disparities among per pupil expenditures in Texas, forced' 
States to devise alternative systems for funding public education; and 
tihe RIPP seminar considered these. ^ The personnel in the SEA's, it is 
obvious from the presentations, needed a tharouuh9oiiiy knowleuge of 
judicial, legal, and 'accounting] principles if tfiey were to keep on their 
heads the rn&ny new hats being placed there. . - " 

r 

Multiple Issues Stretch SEA'^ Leade rship RpXe 

The 1 974'Rocky Mountain RIPP workshops reveal 'how far afield 'the SEA's 
were requiredvto go in maintaining a leadership stance in publ ic. educa- 
tion. At the February 19-20 conference in Al buquerflue. New Mexico, the 



participants discussed "strengthening communications at all levels." ;. 
With so many clients to serve and listen to, the SEA's were concerned : . 
•about improving their ability to communicate— with each other, with the 
legislatures, with the PTA's, with the LEA's, and with special interest 
groups. Oh the horizon were such tentative- concerns as Revising infor- 
mation management systems and management information systems. 

>he seminar held in Las Vegas, Nevada, May 14-15, 1974, zeroed in on 
affective education and evaluation. ■ It was another effort to examine , 
methods of becoming accountable for the product as well as the process 
of education. Behind the topics d1scussed-T"Chi Id-Centered Evaluation, 
"Statewide Eva>uat,ion: ' New Mexico Style," ".Information System for 
Decision-Making," ^'Behavioral Objectives in Perspective" and "Measure- 
ment" in Support of Affective Educat1on"--lay two .very intense questions : 
what is the proper fbcus of the school in assessing child development, 
and how do you translate values and attitudes into educational Objec- 
tives which can be measured? The wr,itinq of behavioral objectives, it- 
self so much a part of an accountabi 1 i^ schema r had apparently be- 
come a particularly noisome task which Targely eliminated the affective 
domain. The abstractions wrestled with at this seminar perhaps reflect 
some frustration that the reach of education was exceedi(>g its grasp, 

f 

Neva>da Depar^ent of Education Sponsorship 

In 1974 the Nevada Department of Education assumed the sponsarstiip of 
the Rocky Mountain Regional Interstate Planning Project Program under the 
direction of Lamar Lefevre Assistant Superintendent of Public Instruction 
The progVam in Boise, Idaho, August 12-14, addressed community education, 
and once again U was clear that the schools had to learn to reach out 
~to more publics. Among the questions explored were thg^e: /What is the 
• concept of community education? What is the role of the SEA in community 
education? What is the responsibility of 'the local school districts to 
offer community education? What role' should universities and colleges 
play in community education? 

Despite the fact that cotmiunity education threatened the'i^h'ole organiza- 
tional structure of elementary and secondary schools, the presenters 
were imbued with a sense of excitem.ent. Beyond the economies offered 
by community schools was t^ie ferv£nt hope that the school and the com- 
munity could once again' unify as they once had in village life. Hope 
Kading, Vice Chairman of Aihe Education Commission .of the States, ex- 
pressed it this way: ; ' 

The secrety to mey of coTrmuhity adiLC<ftion. . .lies in our 
ancient. . .concept of .commnity. We must actually view the 
schoolhduse from all points in the circUy^like a medicine 
'Wheely bel^auSe I believe that'^ the, only way we're going to 
recover something that I find. : .crucially important—the 
sense \>f ^corrmmity. ' • ' ^ ' \ 




I 



The next seminar at Salt Lake City^Utah, December 10-12., 1974, is a 
testament to' the manifold issues that were marshalling at the doorsteps 
^of educatorsr metric education, career education, exceptional children, 
ed^ucation'of the poor, ed<jcational governance. In his sunpary/of the 
coiiference, Kenneth Hansen, Nevada Superintendent of Instruction , aptly 
termed it "a classic example of the Kurt Lewin force, field ana]ysis/' 
By that, he meant that in every session he observed, the participants ^ 
seemed to' be saying, /'Here are the forces going for us. Here .are the 
forces pushing against the solution of this problem^, let's *ort them 
' out and see where we go from there." It was^a highly professional ex- 
' change of views of p-ersons in very diverse fields and levels^of educa- 
* tion,, what'Hansen called "the RIPP mix."- Robert F. Tardiff reported on 
the Math Education Tasfc Force RecOmmfendations for Metric Education, 
Roald Campbell reported on the Educational Governance Project and 
Assistant Superi-ntendent Beatrice M. Smith 6f Texas reported on a pr^- 
l^indargarten screening program in her "State. 

SEA Staff and System^ Renewal Bectmes Promigent RIPP IsSife 

Beginning with the conference in El Paso, Texas, April 1-3, V975, the 
Rocky Mouj;itain RIPP embarked lipon a series of expansive'topics,* seeking 
new. dif?ct1ons and methods of renewal. It wa^a refreshing view outward 
from an embattled educational promontory. Th^ El Paso meeting focused^ 
upon the role of SEA*s in international community education. An infusion 
of provocative ideas from Dr. Richard Bedggood, President of t>ie New 
Ze^and Community Education Asspfiation; from Dr. Shannon Hagan, Educa- 
tional Officer of the Ministry of Education in Ca^nada; from Dr. William 
Martin, Senior Program Advisor for International Education, USOE; and 
from Virginia Trotter , Assistant Secretary for Education, HEW, created a 
sense of shar^ i:ommunal purpose across cuHures and geography. Trotter 
said fitting'ly: ''I am-convinced that the time- has come vyhen we are .talk- 
ing about renewed 'hopes and ethics and spiritual resources -among our cit- 
izens, our institutions, and in our communities at home and abroad." The 
. idea of commutiity education was perceived as extending. globalTy. 

.For the nextJ^IPP seminar held in Heleija, Montafra, July 1 -3, 1975, the 
Board of Directors returned briefly to the fray and chose once again a 
familiar topic, accountability, but with a different emphasis. The pub- 
lic demand for educational accountability had bred a basic need for infor- 
mation to support decision making^'and justify policy. Procuring that ^ 
^ information on the Most timely, cost-efficient basis had become a topi*c 
of intense and continuing interest to the^ RIPP consortium of States, 
they needed ttf^ know-how to estd4)lish or improve Educational Management^ 
Information Systems. , ^ 



The state. of the EMIS art was thor9ughly explored at this meeting. Each 
State was represented by a team .cohsistin^of staff from three levels: 
top management or policy making. State EmR director^or administrator, 
and EMIS analyst or technician. EMIS^was defined, and ways to design 
systems which fit actuaV information needs were discussed. *&ut, on the 
fringes or within the depths of each address to the conference ley a 
sense of frustration that EMIS^s weren't accomplishing all that they 
should. The language of the data analyst and data processor invested 
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the discu^sior with the aura of technology rampant, but within the fric- 
tion^-of debate between tte usc^s and producers of' information , a sense 
of common i^urposer-to use^ these sj^stems- to help solve maj.or educational . 
problems^-was attained. - , 

Aniong-more »|Dtable presertters at the conference were :Dr. James Mitchell, ; 
Associate Superintendent of thi Iowa Department of Rublic Instruction; 
J. F. Movizzo of the Data Processing Division, IBM; Thomas M. Nies, Pres- 
ident of CFNCOM Systems/lncorporated; and Mrs. ,Katherine Wal lhan , Na- 
tional Center for Educational Statistics. ^ ^ ^ 

If the RIPP confdfees were asked to look outward in El Paso^, tliey were 
askerto look inward as well at the next seminar a^Lake Tahoe, Nevada, 
November 12-14, 1975. The topic was "education adrimiistrative renewal, 
and the participants sought fresh direction and renewed vigor in coping 
with the many problems which beset them. -Administrators were shewn how 
to confront without having to combat by working througrh collegial systems 
for professional growth--and the emphasis was not on the system, but on 
tte human element. The intr-usion of so many compelling social issues 
into the Schools-^racial balance, sexual equity, student rights--strained 
the administrators' ability to resp^ond. As Nevada Deputy Superinten- 
dent of Public Instruction James Costa saw it, "ualess the administrator 
ca.n be offered opportunities to develop skills and insights, to provide 
leadersfvip in educational planning and 'directing , Ifnanagement by crisis 
will become standard operating procedure." - 

Again, at this confere>Krer^the emphasts was placed on '"active optimism." 
Educational management, like corporate management, was presented as a 
pe*rf«ctl^ workable process for resolvjng problems. , The presenters were 
a distinguished group "of professionals/ Among them were Dr.. James E* 
Conner, Director of the Staff Development Project, Council of Chief 
State School Officers, who spoke on educational riianagement in terms of 
people and organization renewal; Dr^ Patrick E. Connor, Associate Pro- 
fessor of Management at Oregon State University, who discussed individ- 
ual versus group problena-soTving processes; and Dr. Jerrence Deal, Re- 
search* Associate , Stanford Center for Research and Development Training* 
who (liscussed "productive do^IJision" as a means of admirvi strati ve renewal 

*rhe next conference at Phoenix, Arizona, February 17-18, 1976., extended 
the exaniination of administrative renewal to school district personnel. 
As in previous conferences, the attempt Vas made to get down to specific 
cases* Individual programs wfere described and different approaches to 
help administrators become %etter educational leaders were examined* To 
round, out the program Dr. James Pahlau, Director of Instruction^' Cannon 
Cit^ Colorado, dis^tessed "Administrative Renewal in- a Small School • 
• bistrict." Dr. Rich>R^. Marr, Director of Personnel in Alvoicd, Cali- 
fornia, discussed "Administrative Renewal As Bart ofan Intermetftate- 
School Model"; and Dr. Charles Hough, Principal Of IrtTaham High School 
in Seattle', Washington, discussed '^Administrative ReT^wal in an Urban I 
Area: from the 60>.to\Tie 70's." The participants ^ere giyen the J nuts 




*and bolts"' approach to motivating staff and rawing self. The feeling 
of optimifim, evident in ^e prior conference, prevailed at this one and 
might be typified by two comments, one by Dr. William D. Hawkins, Super- 
intendent of the Brighton, Colorado School District, "The alternative to 
progress 'Is decay;" and the other by James Klassen, Roseville Area 
Schqols, St. Paul, Minnesota, "If you think you are beaten, you are." 

Complex Issues in Evaluation and Bilingual Education Explored ' 

When the RIPP consortium met again in Denver, Colorado, July 21-23, 1976, 
the topic chosen for investigatioh- was "evaluation for effecting educa- 
tional futures." Although evaliiation is basic to accountability, a fa- 
mlliar subject explored In many previous RIPP conferences, it was diffi- 
cult for the conferees to get a firm handle on it. Evaluation evidently 
meafls quite different things to different people. Dean Arthur Coladarci , 
Stanford University, sunmed.it up in the tidy^phrase "sacrosanct^ and 
profane." > 

The scope of this conference ranged broadly from Dr. Launor Carter's 
comprehensive review of the National Study of Title I projects being con- 
ducted by System Development Corporation, to a discussion of how -nfor- 
mation is utilized in decision-making by Dr. Roy Forbes, ;)i rector of Na- 
tional Assessment of Educational Programs, to presentations o1| several 
ki,nds of evaluation efforts by representatives of various SEA s. I he 
addresses and discussions were rich in ideas, and theVprogram was a 
very full one. The four objectives of the conference— to identify prob- 
able directiorvs of change, to share experiences and ideas on evaluation 
activities, to review promising or emerging evaluation practices, and to 
secure concrete suggestions on utilization strategies for renewing or 
recycling educational programs--were admirably achieved, according to 
the workshop evaluation comments. ■ , 

Th^nfixt Rocky Mountain RIPP conference in San Antonio, Texas, November 
17-19, 1976, confronted another topic particutarly difficult to securely 
gri^p because of the legal, linguistic, cultura^and educational rami- 
fications: -bilingual education. It is a tribute to the conference 
planners that they were able to assemble such a remarkable array of pro- 
fessioTial" and, scholarly talertt to pursue the implications of bilingual 
" education so intensively. 

The States of Texas, Arizona, and' New Mexico , with their lengthy experi- 
ence i<n biligual education progr*ams, provided a keen, informed perspec- 
tive for. seminar discussioas. Dr. Robert Acosta, Director of the Post- 
-secondary Division ^f the Office of Bilingual Education gave a general 
overview of federal involvements,*. Henry PascuaU Director of the Bi- 
lingual Teacher Training Unit; New Mexico Department of Education was 
warmly received fof his provocative discussion of "fifteen years in bi- 
lingual/multicultural education." The implications' of the Lau decis]on 
were thoughtfully explored by Jim Littlejohn, Associate Branch .Chief , 
Office for Civil Rights, Region VI; and Dr. Ed DeAvila's paper on lan- 
^quage assessment, cognition, and the Lau decision provided , a- flrm schol- 
arly perspective.' Dr. S. Gabe Paxton, Jr., Associate Deputy Comniss'ion. 
er In the Office of Indian Education, skillfully surveyed the probleias 
of Indian blUngual education through many tribal projects. 



This workshop, one of* the last, was surely one of the best. It demon- 
strated the success of the Rocky Mountain RIPP fomat— "the RIPP mix." 
Representatives af federal, regional; Ctate, and 1^-^ educational levels 
met and investigated a complex topic of intense i:oncern, they were ex- 
posed- to abstract theory and concrete applicertrrcn, and they leftwith a 
much fuller knowledge of what bilingual education entailed and what they 
must work toward in the future. 



Summary^ v 

During the nine years that the Rocky Mountain Regional Intersta^ Plan- 
ning Project has been in existence, it has certainly fi/lfilled its ini- 
tial purpose: to strengthen the leadership of the participating State 
departments of education. It has gone far beyond that, engaging the in- 
terest and support of regional offices and the USOE as wfell. * It has 
proved to be a remarkably flexible forum, able to respond effectively to 
the various new demands confronting public education. 

The Issue of accountability has been a steadily recurring one. The re- 
lated topics of management by objectives, information management • 
evaluation, and performance-based teacher certification were fully e/- 
p^ored. .It is apparent from tne greater sophistication of questions 
^ posed and issues addressed that the SEA's have" used the RIPP confer-^ 
^ences to acquire or refine leadership skills, A healthy skepticism of 
systems or theorieS^hich offered shallow evidence of workability often 
appears in evaluation commentary. And each succeeding seminar has strived 
to incKWe practical application. 

Yet the forum has also reached often endugh for those elements Which 
nurture the spirit ^f education. Systems have not been given sway over 
concerns about children. Superintendent Jim Miller of Santa Fe, N.ew 
Mexico, poetically posed the question for bilingual education: 

They bring a people's native pride and fp^ theit 'Zult^e is 
denied. Whp wilt answer? 

The same concern was evident in discussions about community education, 
affective education, mainstreaming the handicapped, educational equity, 
and administrative renewal. 

The RIPP organization has been thoroughly professional in its operation. 
For presenters, it has sought and secured the services of highly informed 
individuals within and without the educational communily--legislators, 
corporate executives, professors, local school district personnel , 'teach- 
er union representatives, federal and State. officials, researchers, and 
attorneys. Becairse of the careful planning of the workshops, the^ part'i- 
"cipat'lng SEA's have been exposed to a full exchange of ideas on many of 
the most pressing arid troublesome issues in education. Consequently the ' 
Roc ky/^ Mountain Intersta*te Pl.anninq Project has evolved into a valuable 
conduit ^rough which SEA^s can see^k and share experience and knowledge. 
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Seminar Evaluati on Form 
I. DEMOGRAPHIC : ' ^ , 

1. Professional Position (Check One)-: 
4 local district staff 



35 state education agency *staff 

0 school board^ 

0 university staff 

6 other: 4 eYS 



•] ' Regional Education Serv.ice Center 

1 Intermediate Uni t 



"2.. What State do you represent? 

3. Overall , >i n comparison with other educational workshops you have 
attended, how would you rate this one? 

5 Top 10' - -6 . ^ 2 L6wer 50% - 2 



4 -- Top 20' - 22 ] Lower 20'-, - 2 



3 -- Top 50" - n ■ ' 0 -- Why Bother - 0 

Added by conference participant - 6 -- Top Z% - 1 

• S EMINAR OB^JECT IVES 

'*11 participants will become aware of and better informed about 
the California Evaluation Improvement Project an^ its approach 
to evaluation. 

Participants who are program managers from state and inter- ** 
mediate levels and*those who are already familiar with research 
and eval uatioH' concepts will be able to describe the project to 
others and to acQuire skills required to present the materials 
in Sut)sequent workshops. 

Pa rti^fipants fi'om ^ocal school districts will acquire skills re- 
Quired for implementing or improving evaluation designs in their 
own districts. 

0 



II. OBJECTIVES: 



4. -'Were, you able to identify probable directions in vthich evaluation could 
assist with the change process? ' 

positively yes, generally/ only somewhat not at all 

20 22 3 -0- 
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Wer^ you 'able to share experiences and ideas^ on .ev^l uation activities and 
findings'? • * r ' ' 



positively 
12 



yes, generally 
25 



only somewhat 
7 



Wererybu provided the opportunity to review promising or em 
modeils and practices'^ 



not at all 
0- 

eval uation 



Ti^rg i ng 



^ positi^ 



positively . yes, generally only somewhat 

13 " 11 . > 18 



not at al 1 



Did you pain ideas and suggestions that couM be useful in your situation- or 
responsibility? 



positively 

23 



yes, generally * only somewhat not at all 

15 7 • -0- 



^ In general, how apolicable were workshop acti vi ties ^to your local situation? 



h 1 c h y 
aop'l 1 cabl e 

1? 



neneral ly 
apDl 1 cabl e 

31 



not 

appl icable 



no 
comment 



III. PHYSICa ARRAN^^EMENTS: 



Please Co^pen^ : 



No Preble^ • 

Seatmq Stvle was Cunberson 
Ver/ inconvenient ^or A^ter 

VJork Activit^'es 
Fine ^ 
Excellent Acconnoddtions 
Super 



5 
17 
4 



fluiet Favorable 
Passable 
Very Rood 
Ni ce 

Hats off to 
Comfortable 
No Cpmmeat 



Showbtoat Staff 
and Adequate 



1 
1 
6 
2 
1 
1 
1 



IV. WORKSHOP Pt^ESENTOPS- 



Well Prepared and Presented - 2^- 

Missing in Tone - 8'" 

^ood Practical Approach - 5 

Good/Fair - 5 

No' Comment - 



Added Comments by Participants : 

Well Endowed - 1 
Poor - 2 



V . GE^E^AL_CJWM_ENTS_A_BOU^ W_Om<SHOP : 

Presentors Tbok Too Much Time 
- ' Presentors Should Have Worked With Group 

Some Components More Useful than- Others 
Wednesday Morning Activity was/Interesting 
Too >UiCh Material - Not Presented Well 
' * More Participation from Groups 

Thanks for Opportunity of Sharing Ideas with Others 
Renewal to Me - Job Well Done 
/ More Emphasis on Exercises ^ 
Q {jgre Silent Reading and Less Lecturing 
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3 
2 
■1 
1 
1 
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V . GENERAL COM MENTS _ABilUl T H E_ WORKSHOP - C_Qn_tj n ued 



One of The Better 

Excellent 

Very Useful 

Pleasure Watching Richard Harsh Work 
Felt A LUtle Rushed 

Should Have Had Simulatior^ Games Earlier 
Needs To Be Structured 
No Comment 

V I . DID YOU ENJOY YOUR STAY IN LAS VEQAS, _N_E V A^D A : 



Yes 

Yes, But I'lTi Broke 

Yes, Hood Hosting 

Yes, But Expensive , 

WOW 

Too Short 
So, So 

Going Back To Work To '^est 

Immensely 

Very Much 

Def ini tely 

No Conment 



- 20 

- 2 
> 4 

- 3 



2 
2 
7 



